[GPU] sdpa_micro for prefix caching #31968
Merged
+543
−266
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
e-ddykim
force-pushed
the
sdpa_micro_pa
branch
2 times, most recently
from
d5b9a06 to
a62e2ad
Compare
e-ddykim
force-pushed
the
sdpa_micro_pa
branch
3 times, most recently
from
5da134a to
d95932b
Compare
e-ddykim
force-pushed
the
sdpa_micro_pa
branch
from
c330023 to
98cb9b2
Compare
github-actions
bot
added
the
category: transformations
yeonbok
reviewed
Oct 13, 2025
| } | ||
| } | ||
|
|
||
| if (config.is_paged_attention && data_type_traits::is_i8_u8(K.data_type)) { |
There was a problem hiding this comment.
can't we use config.is_kv_compressed?
There was a problem hiding this comment.
The config.is_kv_compressed is being used for the non-PA case. I'm not sure when it is used. But from the code, I see that it requires separate scale and zp inputs when config.is_kv_compressed is set. So, I didn't config.is_kv_compressed for the PA case.
yeonbok
reviewed
Oct 14, 2025
e-ddykim
force-pushed
the
sdpa_micro_pa
branch
from
98cb9b2 to
63db02c
Compare
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
e-ddykim
force-pushed
the
sdpa_micro_pa
branch
2 times, most recently
from
94065db to
aea9c4c
Compare
github-actions
bot
removed
the
category: transformations
e-ddykim
force-pushed
the
sdpa_micro_pa
branch
from
aea9c4c to
87deb35
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Details:
sdpa_microto support paged attention for better performance.mixedstage of paged attention will be handled bysdpa_microinstead ofpa_sdpa_opt.sdpa_microto supportsliding window.Tickets: